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Abstract 

The  multiattribute  utility  copula  is  an  emerging  form  of  utility  function  used 
by  decision  analysts  to  study  decisions  with  dependent  attributes.  Failure  to  prop¬ 
erly  address  attribute  dependence  may  cause  errors  in  selecting  the  optimal  policy. 
This  research  examines  two  scenarios  of  interest  to  the  modern  warfighter.  The  first 
scenario  employs  a  utility  copula  to  determine  the  type,  quantity,  and  altitude  of 
UAVs  to  be  sent  to  strike  a  stationary  target.  The  second  scenario  employs  a  utility 
copula  to  examine  the  impact  of  attribute  dependence  on  the  optimal  routing  of 
UAVs  in  a  contested  operational  environment  when  performing  a  search  and  destroy 
mission  against  a  Markovian  target.  Routing  decisions  involve  a  tradeoff  between 
risk  of  UAV  exposure  to  the  enemy  and  the  ability  to  strike  the  target.  This  re¬ 
search  informs  decision  makers  and  analysts  with  respect  to  the  tactics,  techniques, 
and  procedures  employed  in  UAV  search  and  destroy  missions.  An  ever  increasing 
UAV  operations  tempo  suggests  such  research  becoming  increasingly  relevant  to  the 
warfighter. 
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Using  Multiattribute  Utility  Copulas  in  Support 


of  UAV  Search  and  Destroy  Operations 


1.  Introduction 

1.1  Background 

The  UAV  Roadmap  2005-2030  states  that  information  gathering  has  histori¬ 
cally  been  a  dangerous  mission  and  as  a  result  UAVs  have  become  one  of  the  pri¬ 
mary  mediums  for  information  gathering  activities  in  modern  warfare.  The  UAV 
has  proven  itself  to  be  an  effective  tool  for  not  only  information  gathering  but  air  to 
ground  engagements  as  well.  Indeed,  the  UAVs  has  proven  to  be  better  suited  for 
“dull,  dirty,  or  dangerous”  missions.  As  a  result,  UAVs  have  become  an  integral  part 
of  planning  activities  in  US  operations,  and  substantial  funds  have  been  provided 
for  the  continued  development  and  integration  of  UAVs  into  National  Defense  [1,  2], 

Employing  UAVs  effectively  on  the  battlefield  requires  proper  planning  and 
decision  making  [2],  Although  continuous  coverage  over  all  areas  of  interest  is  ideal, 
limitations  on  the  number  of  UAVs  available  to  commanders  and  vulnerabilities  to 
enemy  anti-air  activity  make  this  infeasible.  A  fully  operational  Predator  system 
consists  of  four  aircraft,  a  ground  control  station,  a  satellite  link,  and  crew  [41], 
Typically,  12  Predator  systems  are  spread  among  three  squadrons  [1].  With  those 
limited  assets,  a  commander  must  decide  where  to  send  the  aircraft  with  what  pay- 
load.  The  analysis  presented  in  this  thesis  assumes  the  commander  seeks  to  maxi¬ 
mize  the  number  of  enemies  neutralized  while  simultaneously  minimizing  number  of 
friendly  UAVs  destroyed.  Defense  experts  state  that  UAV  assignment  is  critical  to 
the  success  of  US  intelligence  operations  [1]. 
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This  thesis  presents  an  analysis  of  the  decision  situation  faced  by  a  comman¬ 
der  assigning  UAVs  to  information  gathering  activities  and  air  to  ground  missions. 
The  motivation  is  to  examine  the  tradeoffs  between  achieving  enemy  disruption  and 
exposing  friendly  aircraft  to  risk  in  an  uncertain  environment.  Multiattribute  Utility 
Theory  (MAUT)  provides  a  clear,  transparent  framework  for  investigating  the  issues. 
The  research  objective  is  to  develop  a  decision  model  for  commanders  to  use  in  the 
assignment  of  UAVs  to  targets  that  will  improve  the  ability  of  the  armed  forces  to 
counter  asymmetric  warfare  operations,  enhance  global  efforts  in  fighting  terrorism, 
and  an  perform  other  US  allied  objectives. 

Studies  concerning  the  management  of  UAVs  in  conflict  have  been  undertaken. 
The  problem  of  UAV  assignment  and  control  is  modeled  as  an  integer  linear  program 
by  Shirna  and  Rasmussen  [36],  a  stochastic  continuous  coverage  problem  by  Ha  [16], 
and  a  simulation  of  large  numbers  of  micro  UAVs  by  Corner  [10].  An  object  oriented 
simulation  to  examine  the  use  of  UAVs  in  surveillance  and  active  suppression  of 
enemy  air  defenses  was  conducted  by  Walston  [43].  The  use  of  UAVs  in  special 
operations  is  examined  by  Howard  [19]  and  aerial  refueling  techniques  for  extended 
UAV  operations  by  Stephenson  [38].  Longino  [25]  gives  a  thorough  history  of  the 
development  and  use  of  UAVs  from  World  War  II  onward.  Kennedy  [23]  presents  a 
plan  for  the  future  development  of  UAVs,  specifically  focusing  on  the  development 
of  High  Altitude  Endurance  UAVs. 

The  distinguishing  feature  of  this  research  is  the  ability  of  a  commander  to 
express  risk  aversion  or  risk  seeking  attitudes  in  the  process  of  assigning  UAVs  to 
targeting  operations.  It  approaches  the  assignment  of  UAVs  to  missions  using  de¬ 
cision  analysis,  the  result  of  which  is  the  development  of  a  utility  function  linking 
enemy  capability  and  the  number  of  UAVs  to  assign  to  a  mission. 
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1.2  Problem  Statement 


A  commander  is  often  faced  with  decisions  with  competing  objectives.  The 
commander  has  a  desire  to  maintain  the  overall  readiness  of  the  fleet  of  aircraft 
by  not  unnecessarily  losing  UAVs,  while  simultaneously  neutralizing  enemy  threats. 
This  decision  forces  a  commander  to  examine  the  risk  attitudes  associated  with  UAV 
loss  and  enemy  disruption  as  well  as  tradeoffs  between  these  two  attributes.  This 
research  explores  these  tradeoffs  in  a  multiattribute  decision  model  that  provides  the 
framework  for  optimal  decisions  to  be  made  under  uncertainty. 

1.3  Overview 

Chapter  2  presents  the  framework  for  creation  of  a  multiattribute  utility  deci¬ 
sion  model  and  proceeds  to  solve  a  decision  using  the  model.  The  first  model  involves 
a  stationary  target  and  a  set  of  2  UAVs  available  to  assign  for  sensor  or  strike  duties. 
Chapter  3  provides  a  more  complex  model  that  introduces  time  dependent  searches 
for  a  moving  enemy  using  a  fleet  of  2  UAVs.  This  model  allows  for  Bayesian  updating 
of  the  decision  space. 
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2.  An  Application  of  Copulas  to  UAV  Mission  Planning 

2. 1  Introduction 

Most  decision  analysis  problems  are  modeled  with  a  multiattribute  utility  func¬ 
tion  combined  with  a  joint  probability  distribution  for  the  prospects  of  a  decision 
situation  [4],  When  facing  a  situation  where  the  variables  are  independent,  this  joint 
distribution  is  greatly  simplified,  becoming  the  product  of  their  marginal  distribu¬ 
tions. 

Similar  to  joint  probability  functions,  the  presence  of  utility  independence  be¬ 
tween  attributes  in  a  decision  situation  greatly  simplifies  the  construction  of  a  multi¬ 
attribute  utility  function  [4] .  When  constructing  a  utility  function  over  the  attributes 
x  and  y  where  there  is  mutual  utility  independence,  the  multiattribute  utility  func¬ 
tion  is  of  the  multilinear  form  [22], 

The  construction  of  utility  functions  with  dependence  between  the  attributes 
can  be  accomplished  through  several  different  approaches  [4],  One  such  approach 
is  to  construct  a  value  function  that  directly  encodes  the  value  of  the  attributes, 
then  construct  a  single  attribute  utility  function  over  the  value  function.  Since 
the  utility  function  is  reduced  to  a  single  attribute  case,  this  approach  requires 
no  assumptions  of  utility,  preferential,  or  conditional  independence  [26].  Another 
approach  to  modeling  dependence  involves  assumptions  of  preferential  or  conditional 
utility  dependence.  Farquhar  [12]  uses  fractional  hypercubes  to  model  a  higher- 
order  utility  function  as  a  product  of  single  attribute  conditional  utility  functions. 
Bell  [7]  uses  conditional  utility  independence  to  show  that  a  utility  function  can  be 
represented  in  two-attributes  in  an  additive  or  multiplicative  form. 

Most  of  the  recent  work  in  attribute  dependence  focuses  on  direct  assessment  of 
utility  without  assumptions.  One  such  example  of  this  approach  is  seen  in  Kirkwood 
[24],  who  uses  parametric  dependence  (i.e.,  a  dependence  condition  requiring  the  use 
of  an  extra  variable)  to  directly  assess  the  utility  function  whenever  the  conditional 
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parametric  dependence  conditions  hold.  Harvey  [17]  uses  a  concept  of  risk  linearity 
to  derive  a  logarithmic  form  for  a  utility  function  from  a  linear  relationship  between 
values.  Bell  [8]  develops  a  method  for  constructing  conditional  utility  functions 
requiring  no  assumptions.  Keeney  [21]  uses  preferential  independence  to  find  a  set 
of  fundamental  attributes. 

The  fundamental  approach  of  this  research  is  one  of  direct  assessment  of  utility 
dependence.  Since  utility  functions  can  be  built  based  on  the  properties  of  proba¬ 
bility  distributions,  properties  of  probability  distributions  are  exploited  to  build  de¬ 
pendence  structures  in  utility  space  similar  to  those  that  already  exist  in  probability 
space.  This  approach  allows  for  the  construction  of  a  two  attribute  utility  space 
from  the  marginal  utility  of  each  attribute  with  an  assumption  of  non-existence  of 
utility  independence. 

Abbas  [3]  develops  the  multiattribute  utility  copula,  which  links  utility  theory 
and  the  probabilistic  copula  structure.  Abbas  incorporates  an  Archimedean  copula 
structure  and  more  specifically,  Frank’s  copula  [13].  This  structure  allows  for  a  cou¬ 
pling  of  two  correlated  distributions  into  a  bivariate  distribution  using  a  parameter  5 
to  relate  the  dependence  between  the  distributions.  For  more  information  on  copula 
structures,  see  Nelson  [28]. 

In  order  to  utilize  the  copula  structure,  a  measurement  of  statistical  depen¬ 
dence  between  distributions  is  required.  One  such  measure  is  Spearman’s  rank  corre¬ 
lation  coefficient,  Spearman’s  rho  (p);  it  expresses  how  well  the  relationship  between 
two  variables  can  be  described  using  a  monotonic  function  [28].  The  assessment  of 
Spearman’s  rho  is  possible  through  direct  interaction  with  the  decision  makers,  as 
proposed  by  Reilly  [32],  Once  Spearman’s  rho  has  been  assessed,  Genest  [15]  pro¬ 
poses  an  approximation  to  convert  the  value  of  p  to  the  required  parameter  5  used 
in  Frank’s  copula. 

The  purpose  of  this  research  is  to  apply  new  techniques  in  decision  analysis  to 
an  important  UAV  assignment  problem.  This  construction  informs  decision  makers 
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with  respect  to  tradoffs  and  risk  by  exercising  multiattribute  utility  copulas.  This  is 
the  first  research  to  propose  a  direct  assessment  of  utility  dependence  for  translation 
to  6,  the  parameter  controlling  utility  dependence  in  an  Archimedian  multiattribute 
utility  copula. 

The  rest  of  this  chapter  is  organized  as  follows.  Section  2.2  describes  the  de¬ 
cision  space  and  the  methodology  for  constructing  the  multiattribute  utility  copula. 
Section  2.3  contains  the  results  of  a  notional  proof  of  concept  scenario  analyzed  us¬ 
ing  this  approach.  Section  2.4  presents  a  sensitivity  analysis.  Section  2.5  discusses 
conclusions  and  future  work.  Throughout  this  assessment,  the  decision  maker  is  the 
commander  of  a  fleet  of  UAVs  and  the  terms  are  used  interchangeably. 

2.2  Methodology 

The  following  section  contains  the  definitions  and  functions  used  to  create  a 
multiattribute  utility  copula.  Assessment  procedures  for  the  parameters  used  to 
model  a  decision  maker’s  risk  preferences  are  given.  Analysis  of  a  notional  scenario 
demonstrates  the  use  of  the  utility  function.  Section  2.2.1  outlines  the  definitions 
and  functions  used  in  this  assessment.  Section  2.2.2  details  the  parameter  assessment 
methods  for  the  decision  maker’s  risk  preferences.  Section  2.2.3  details  the  notional 
scenario  used  to  illustrate  the  multiattribute  utility  copula. 

2.2.1  Definitions  and  Functions.  In  this  analysis,  it  is  assumed  the  decision 
maker  follows  the  axioms  of  normative  utility  theory  developed  by  von  Neumann  and 
Morgentstern  [42]  and  has  a  multiattribute  utility  function  defined  by  U (x,  y )  over 
attributes  X  and  Y.  Lower  case  x  and  y  denotes  an  instance  of  attributes  X  and 
Y.  Let  a;0  (y°)  and  x*  (y*)  denote  the  minimum  and  maximum  values  of  the  X  (Y) 
attribute.  The  domain  of  the  decision  occurs  on  [a;0,  a;*]  x  [ y°,y *].  The  attribute  X 
is  defined  as  the  number  of  enemy  (or  friendly)  forces  neutralized.  The  attribute  Y 
is  defined  as  the  number  of  operational  UAVs  remaining  after  the  engagement. 
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The  methodology  presented  in  this  Chapter  employs  a  Class  1  Archimedean 
utility  copula,  as  introduced  by  Abbas  [3] .  Further  description  is  warranted.  A  Class 
1  utility  copula  requires  conditional  utility  assessments  at  the  maximum  values  of 
the  component  attributes  and  must  satisfy 

=  diVi  +  bi,  i  =  l,...,n.  (2.1) 

A  Class  0  utility  copula  satisfies 

Co(0,  ...,0,Uj,0,..,,0)  =  ciiVi  +  bi,  1  <i<n,  (2.2) 

and  requires  the  utility  assessments  at  the  minimum  values  of  its  complement.  A 
Class  1  copula  is  used  in  this  analysis  because  there  is  a  guarantee  of  strictly  in¬ 
creasing  marginal  utility  functions  at  the  maximum  margin  for  each  attribute.  The 
Archimedian  functional  form  is  a  Class  1  utility  copula  that  satisfies 


Ci(l,  •••,  l,  vh  1, ...,  1)  =  ciiVi  +  bi,  i  —  1, ...,  n. 


(2.3) 


where  a*  =  a(l  —  /*)  and  6*  =  1  —  a*  =  a/*  +  b.  This  implies  the  same  mathe¬ 
matical  properties  of  a  strictly  increasing  cumulative  probability  distribution  or  a 
strictly  increasing  normalized  utility  function  on  the  domain  [0,1].  This  observation 
allows  for  all  well-known  functional  forms  of  cumulative  distributions  (e.g.  the  Beta 
distribution)  to  be  used  [3]. 

In  the  Class  1  Archimedian  utility  copula, 


C{vx,vy )  =  —a- In 


(X  _  e-S(lx  +  (  1-Za>a0^x  —  e~&^ly+^l~lv^Vy^) 


=- s 


+  b,  (2.4) 
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The  parameters  vx  and  vy  refer  to  the  normalized  utility  functions  for  x  and  y.  The 
parameters  lx,  ly,  a,  and  b  are  defined  by  the  following  equations. 


a  = 


b  =  a-ln  (  1 
o 


o(l  —  lx)  —  1  —  U (x° ,  y*), 

(2.5) 

a(l-ly)  =  1-U(x*,y°), 

(2.6) 

1 

'<5Ja:)(l  —  e~siy)/(l  —  e~5))  ’ 

(2.7) 

(1  —  e~six)(l  —  e~siy)\ 

1  -  e-5  )  ‘ 

(2.8) 

This  approach  requires  the  assumption  that  the  decision  makers  utility  functions  vx 
and  vy  are  exponential  with  risk  aversion  coefficients  7X  and  respectively.  The 
parameter  6  represents  the  trade-off  functions  between  attributes.  An  estimation  of 
6  using  Spearman’s  rank  correlation  parameter  p  is  given  by  Genest  [15]. 


p(a)  ~ 


a^ln(a)  —  a  +  1 
—  l)2 


(2.9) 


where  5  =  —Ina. 

In  order  to  use  this  method  of  assessment,  the  parameters  U(x°,y*),  U(x*,y°), 
p,  7s,  and  7 y  must  be  assessed  from  the  decision  maker. 


2.2.2  Parameter  Assessment.  The  parameters  U(x°,y*)  and  U(x*,y° )  can 
be  directly  assessed  from  the  decision  maker.  In  the  baseline  model,  a;0  is  defined 
destroying  friendly  forces  (thinking  they  are  enemy)  x*  is  complete  neutralization 
of  enemy  forces,  y°  is  the  destruction  of  two  (all)  UAVs,  and  y*  is  survival  of  two 
(all)  UAVs.  An  example  of  the  lottery  for  the  assessment  of  U(x°,y*)  would  be 
(. x°,y *)  ~<  (x*,  y*)]  7ty,  (x0,  y°)  >,  where  U(x°,y*)  =  7Ty.  A  similar  lottery  can  be 
developed  for  U(x*,y°). 

The  parameters  7^  and  7 y  are  the  risk  aversion  coefficients  of  the  utility  func¬ 
tions  vx  and  vy,  respectively.  Steps  for  the  assessment  of  risk  aversion  coefficients 
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can  be  found  in  Keeney  and  Raiffa  [22] .  These  parameters  can  be  obtained  from  the 
decision  maker  through  a  series  of  lottery  questions. 

Reilly  [32]  suggests  that,  given  proper  training  in  variables  of  interest  and  train¬ 
ing  in  the  assessment  method,  rank  correlation  values  can  be  meaningfully  assessed 
by  asking  the  decision  maker  for  an  estimate  of  the  strength  of  the  relationship  be¬ 
tween  attributes.  This  would  be  accomplished  by  asking  the  question,  “How  would 
you  characterize  the  strength  and  nature  of  the  relationship  between  Ux(x)  and  Uy(y), 
using  a  scale  from  -7  to  7,  where  -7  represents  a  very  strong  negative  relationship, 
0  represents  no  relationship  and  7  represents  a  very  strong  relationship?”  In  order 
to  directly  assess  using  this  method,  training  is  required  on  the  definition  of  utility 
dependence  as  it  relates  to  probabilistic  dependence.  This  direct  assessment  method 
is  one  of  the  easiest  to  understand  and  provides  a  consistently  low  absolute  error 
and  low  standard  deviation  in  its  results  [32],  This  step  is  integral  to  applying  the 
multiattribute  utility  copula  and  combined  with  the  Genest  approximation  provides 
the  backbone  for  the  dependence  structure  of  the  multiattribute  utility  copula.. 

2.2.3  Notional  Scenario.  In  this  section,  a  notional  scenario  is  constructed 
to  illustrate  how  the  concepts  of  MUAT  can  be  applied  to  solve  a  decision  problem 
faced  by  a  commanding  officer  of  a  squadron  of  UAVs.  The  rest  of  this  section  is 
organized  as  follows.  Section  2.2.3. 1  presents  the  decision  context  faced  by  a  com¬ 
mander  in  the  area  of  operations.  Section  2. 2. 3. 2  presents  the  information  that  the 
commander  has  at  the  time  the  decision  must  be  made.  Section  2. 2. 3. 3  presents  the 
alternatives  faced  by  the  commander  and  assumptions  the  commander’s  preferences 
for  value  and  risk. 

2.2.3. 1  Decision  Co?itext.  Consider  a  situation  where  a  commander 
has  two  UAVs  able  to  be  dedicated  to  a  mission  in  order  to  neutralize  a  stationary 
enemy  threat.  At  the  beginning  of  the  mission,  the  commander  has  an  intial  belief 


2-6 


in  the  enemy  presence.  The  commander  must  decide  how  many,  if  any,  UAVs  to 
dedicate  to  the  accomplishment  of  this  mission. 


Figure  2.1  Assessed  Decision  Diagram 

2. 2. 3. 2  Information.  At  this  point  in  the  decision  the  commander 
has  an  initial  belief,  P(E)  that  the  enemy  is  in  the  area  and  that  up  to  two  UAVs 
are  available  for  the  mission.  Each  UAV  may  be  sent  with  a  strike  or  sensor  package 
to  determine  if  there  is  an  enemy  presence  and  order  a  strike  if  desired;  however,  if 
a  UAV  package  is  sent,  at  least  one  must  be  loaded  with  a  strike  package  in  order 
to  accomplish  the  mission.  It  is  assumed  that  the  UAVs  have  sensor  capabilities 
that  are  dependent  upon  the  altitude  at  which  they  are  operating  and  the  package 
with  which  the  UAV  is  loaded;  UAVs  with  sensor  packages  are  more  capable  than  the 
UAVs  with  strike  packages  at  detecting  the  enemy.  If  a  strike  is  ordered,  the  proposed 
enemy  threat  is  eradicated  with  a  certain  probability,  depending  on  the  altitude 
and  the  presence  of  a  sensor  UAV.  Each  UAV  acts  as  an  independent  sensor,  with 
independent  results.  Since  the  determination  of  enemy  is  dependent  upon  human 
interaction  in  addition  to  sensor  capability,  this  is  not  an  unreasonable  assumption. 
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In  addition  to  the  sensor  report  obtained  from  each  UAV,  the  loss  of  a  UAV 
can  be  viewed  as  an  update  of  the  probability  that  the  target  is  actually  an  enemy. 
There  is  a  probability  that  a  UAV  fails  without  any  enemy  activity  and  a  probability 
that  the  UAV  fails  due  to  enemy  activity. 

The  decision  diagram  corresponding  to  this  decision  situation  is  given  in  Figure 
2.1.  The  notional  probabilities  associated  with  the  the  decision  situation  are  given 
in  Table  2.1. 


Description 

Probability  Used 

Enemy 

0.7 

Survival  Given  High  Altitude,  Enemy 

0.9 

Survival  Given  Low  Altitude,  Enemy 

0.8 

Survival  Given  High  Altitude,  No  Enemy 

0.98 

Survival  Given  Low  Altitude,  No  Enemy 

0.98 

Sensor  ’’See’s  Enemy”  Given  High  Altitude,  Enemy 

0.75 

Sensor  ’’See’s  Enemy”  Given  Low  Altitude,  Enemy 

0.8 

Belief  that  enemy  caused  failure  given  High  Altitude,  Enemy 

0.9 

Belief  that  enemy  caused  failure  given  Low  Altitude,  Enemy 

0.95 

Sensor  ’’See’s  Enemy”  Given  High  Altitude,  No  Enemy 

0.35 

Sensor  ’’See’s  Enemy”  Given  Low  Altitude,  No  Enemy 

0.2 

Belief  that  enemy  caused  failure  given  High  Altitude,  No  Enemy 

0.6 

Belief  that  enemy  caused  failure  given  Low  Altitude,  No  Enemy 

0.7 

Strike  ’’See’s  Enemy”  Given  High  Altitude,  Enemy 

0.6 

Strike  ’’See’s  Enemy”  Given  Low  Altitude,  Enemy 

0.75 

Strike  ’’See’s  Enemy”  Given  High  Altitude,  No  Enemy 

0.4 

Strike  ’’See’s  Enemy”  Given  Low  Altitude,  No  Enemy 

0.35 

Hit  Probability  Given  High  Altitude 

0.7 

Hit  Probability  Given  Low  Altitude 

0.8 

Table  2.1  Notional  Probabilities  to  be  Assessed  from  the  DM  or 
past  data 


2. 2. 3. 3  Alternatives  and  Preferences.  The  commander  can  send  two 
strike  UAVs,  one  strike  and  one  sensor  UAV,  or  one  strike  UAV  at  high  or  low 
altitude.  Alternatively,  the  commander  may  take  no  action.  Each  alternative  results 
in  a  measure  for  the  attributes  X  and  Y.  The  X  attribute  is  defined  on  [-1,  1]  with 
—  1  -<  1.  A  value  of  -1  represents  a  non-combatant  destroyed  by  the  UAV  and  a  value 
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of  1  represents  enemy  neutralization.  The  Y  attribute  is  defined  as  the  number  of 
UAVs  surviving  at  the  end  of  the  encounter  and  is  defined  on  [0,  2]  with  0  -<  2. 

The  notional  assessed  parameters  for  the  commander  in  this  decision  situation 
are  as  follows.  The  utility  of  a  friendly  kill,  but  all  UAVs  returning  is  U (x°,  y*)  =  0.2. 
The  utility  of  a  succsessful  strike  but  both  UAVs  lost  is  U(x*,y° )  =  0.3.  The  assessed 
ranked  correlation  coefficient  is  p  =  0.7.  The  respective  risk  aversion  coefficients  are 
73  =  —0.2  and  7y  =  0.3.  The  risk  aversion  coefficients  reflect  a  decision  maker 
who  is  risk  seeking  in  gains  and  risk  averse  in  losses  as  prescribed  by  Tversky  and 
Kahneman  [40]. 

After  assessing  these  parameters,  the  multiattribute  decision  copula  parameters 
S,  lx,  ly ,  a,  and  b  are  determined  using  Equations  2.5,  2.6,  2.7,  2.8,  and  2.9.  The 
following  values  are  obtained: 

5  =  -5.0899,  a  =  1.0084,  b  =  -0.0084,  lx  =  0.3058,  ly  =  0.2066. 

The  multiattribute  utility  copula  C(x,y)  is  found  by  substituting  into  Equation  2.4, 

i  qi  /  (l_  e0.31— 0.69[4(a:)\  H  _  e0.207-0.713f/y 

- r^s - 

Figure  2.2  shows  the  multiattribute  utility  function  and  Figure  2.3  shows  the 
isopreference  curves. 
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Attribute  X 


Isopreference  Curves 


Figure  2.2  Multiattribute  utility 
surface 


Figure  2.3  Multiattribute 

isopreference  curves 


2.3  Results 

The  alternative  with  the  highest  expected  utility  is  to  send  one  strike  UAV  at 
high  altitude.  The  likelihood  of  misdiagnosing  the  enemy  presence  is  low  enough, 
due  to  the  high  prior  probability  of  enemy  presence,  that  benefit  of  sending  a  sensor 
UAV  is  outweighed  by  the  risk  of  a  sensor  UAV  being  destroyed.  For  this  reason, 
sending  the  strike  at  higher  altitude  is  preferred  as  well.  Table  2.2  shows  the  resulting 
expected  utility  of  the  5  best  alternatives. 


Rank 

Sensor  UAV/Altitude 

Strike  UAV/Altitude 

Utility  Copula 

1 

None 

One  High 

0.688 

2 

One  High 

One  High 

0.688 

3 

One  Low 

One  High 

0.687 

4 

None 

One  Low 

0.684 

5 

None 

Two  High 

0.681 

Table  2.2  Five  best  alternatives  by  utility  copula 


It  is  interesting  to  note  that  the  multilinear  model  (discussed  by  Keeney  and 
Raiffa  [22])  provides  a  different  order  for  the  solutions.  Not  using  any  dependence 
structure  would  have  resulted  in  the  wrong  decision  being  made.  Since  the  multi¬ 
linear  utility  requires  the  assumption  of  conditional  independence  among  attributes, 
the  resulting  utility  in  one  attribute  would  not  update  the  utility  of  the  other.  The 
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copula  form  allows  for  a  dependence  assumption  between  the  attributes,  thus  the 
utility  of  the  number  of  enemy  destroyed  can  change  based  on  the  utility  of  the  num¬ 
ber  of  UAVs  returning  to  home  base  after  the  engagement,  which  more  accurately 
reflects  the  commander’s  preferences.  The  result,  shown  in  Table  2.3,  is  that  the 
top  four  alternatives  switch  in  order.  The  utility  of  the  intermediate  outcomes  has 
changed  so  that  the  need  to  more  accurately  identify  the  enemy  outweighs  the  the 
risk  of  UAV  loss  therefore  the  strike  UAV  must  fly  at  low  altitude  (with  a  higher 
sensor  accuracy). 


Rank 

Sensor  UAV/Altitude 

Strike  UAV/Altitude 

Utility  Copula 

1 

None 

One  Low 

0.697 

2 

One  Low 

One  High 

0.696 

3 

One  High 

One  High 

0.696 

4 

None 

One  High 

0.694 

5 

None 

Two  High 

0.684 

Table  2.3  Five  best  alternatives  by  multilinear  utility 


2-4  Sensitivity  Analysis 

This  section  details  the  sensitivity  on  the  assessed  parameters  for  constructing 
the  multiattribute  utility  copula.  Section  2.4.1  details  the  sensitivity  to  the  parame¬ 
ter  p  with  an  explanation  of  how  this  parameter  can  change  the  optimal  alternative. 
Section  2.4.2  contains  the  sensitivity  analysis  of  the  model  to  U (a;0,  y*)  and  U (x*,y°). 
Section  2.4.3  details  the  sensitivity  of  this  analysis  to  the  parameters  7^  and  yr 

2-4-1  Sensitivity  to  p.  The  parameter  p  is  translated  to  5  (for  use  in  the 
copula  by  the  Genest  approximation  in  Equation  2.9).  The  parameter  5  defines  the 
corresponding  tradeoffs  curve  between  x  and  y.  Because  the  structure  used  is  a  Class 
1  utility  copula,  the  maximal  marginal  utilities  are  assessed  and  used  to  build  the 
utility  function.  This  results  in  no  change  to  the  maximal  marginal  utilities  despite 
any  changes  to  p.  Changes  in  the  parameter  p  will  result  in  changes  to  the  shape 
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of  the  center  of  the  utility  copula.  Figure  2.4  shows  the  utility  isopreference  curves 
with  p  =  —0.5  and  Figure  2.5  shows  the  utility  isopreference  curves  with  p  =  —0.9 


Isopreference  Curves 


Isopreference  Curves 


Figure  2.4  Isopreference  with 
p  =  -0.5 


Figure  2.5  Isopreference  with 
p  =  —0.9 


The  optimal  decision  changes  between  p  =  —0.6  and  p  =  —0.7.  The  decision 
is  still  not  the  same  as  the  one  made  by  no  utility  dependence  structure,  in  the 
multilinear  model.  This  change  occurred  because  the  utility  of  points  assessed  away 
from  the  maximal  margins  changed  enough  to  affect  a  change  in  expected  utility 
derived  from  the  probability  distribution  of  outcomes.  The  result  is  that  it  is  the 
need  to  positively  identify  an  enemy  outweighs  the  risk  of  loss  to  a  sensor  UAV. 
Table  2.4  shows  the  changes  in  the  decision  based  on  perturbations  in  p.  In  Table 
2.4  Se/St  represents  the  number  of  sensor  and  strike  UAVs  sent  and  the  altitude. 
For  example,  1H/1L  represents  one  sensor  package  UAV  sent  at  high  altitude  and 
one  strike  package  UAV  sent  at  low  altitude. 


2-4-2  Sensitivity  to  U(x°,y*)  and  U(x*,y°).  The  parameter  U(x°,y*)  de¬ 
fines  the  utility  of  the  consequence  in  which  all  UAVs  remain,  but  a  non-enemy  is 
destroyed  (i.e,  collateral  damage  occurs).  The  assessed  parameter  has  a  value  of  0.3. 
Table  2.5  shows  the  top  five  optimal  alternatives  as  a  function  of  U(x°,y*). 
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p=-0.5 

p=-0.6 

p=-0.7 

p=-0.8 

p=-0.9 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

m/m 

0.691 

1H/1H 

0.689 

N/1H 

0.688 

N/1H 

0.687 

N/1H 

0.686 

N/1H 

0.690 

N/1H 

0.689 

1H/1H 

0.688 

1H/1H 

0.687 

1H/1H 

0.686 

1L/1H 

0.689 

1L/1H 

0.688 

1L/1H 

0.686 

1L/1H 

0.685 

1L/1H 

0.684 

N/1L 

0.687 

N/1L 

0.686 

N/1L 

0.684 

N/1L 

0.682 

N/1L 

0.681 

N/2H 

0.682 

N/2H 

0.681 

N/2H 

0.681 

N/2H 

0.681 

N/2H 

0.680 

Table  2.4  Sensitivity  to  p 


U  (x°,  y * 

0=0.1 

U(x°,y* 

:)=0.2 

U(x°  ,y* 

= ) =0.3 

U(x°,y* 

:)=0.4 

U  (x°,  y * 

:)=0.5 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

S/St 

U 

N/1H 

0.681 

N/1H 

0.684 

N/1H 

0.688 

1L/1H 

0.697 

1L/1H 

0.707 

1H/1H 

0.675 

1H/1H 

0.681 

1H/1H 

0.688 

1H/1H 

0.695 

m/m 

0.702 

N/1L 

0.672 

N/1L 

0.678 

1L/1H 

0.686 

N/1H 

0.691 

N/1L 

0.698 

N/2H 

0.668 

1L/1H 

0.676 

N/1L 

0.684 

N/1L 

0.691 

N/2L 

0.696 

1L/1H 

0.667 

N/2H 

0.675 

N/2H 

0.681 

N/2H 

0.687 

N/1H 

0.695 

Table  2.5  Sensitivity  to  U(x°,y*) 


The  optimal  alternative  changes  based  on  sensitivity  to  U(x°,y*)  between  0.3 
and  0.4.  This  parameter  drastically  changes  the  marginal  utility  on  the  upper  mar¬ 
gin  for  y,  resulting  in  a  pronounced  change  in  the  curvature  of  isopreference  lines. 
Figure  2.6  shows  the  isopreference  curve  for  a  U(x°,  y*)=0.1  and  Figure  2.7  shows 
the  isopreference  curve  for  a  U(x°,y*)= 0.5.  These  figures  clearly  show  that  many  of 
the  outcomes  of  the  decision  situation  would  have  drastic  changes  in  utility  based 
on  perturbations  to  U(x°,y*). 

The  parameter  U(x*,y°)  is  the  utility  of  the  consequence  in  which  all  enemy 
forces  are  neutralized,  but  the  two  UAVs  are  lost.  In  the  notional  model,  this  pa¬ 
rameter  has  a  value  of  0.2.  Table  2.6  shows  the  top  five  optimal  alternatives  as  a 
function  of  U(x*,y°). 

The  result  of  changing  the  parameter  U(x*,y°)  was  the  opposite  of  changing 
U(x°,y*).  A  change  from  the  baseline  occurred  between  0.1  and  0.2,  for  similar 
reasons  as  the  change  in  U(x°,y*)]  the  two  parameters  control  a  great  deal  of  the 
overall  shape  of  the  utility  function.  This  change  is  indicative  of  the  higher  utility 
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Isopreference  Curves  Isopreference  Curves 


Figure  2.6  Isopreference  with 

U (x°,  y*)  =  0.1 


Figure  2.7  Isopreference  with 
U(x°,  y*)  =  0.5 


U(x*,y°)= 0.0 

U{x*,y°)=0.1 

U(x*,y°)= 0.2 

U(x*,y°)= 0.3 

U{x*,y°)=0A 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

1L/1H 

0.631 

1L/1H 

0.659 

N/1H 

0.688 

N/1H 

0.723 

N/1H 

0.758 

1H/1H 

0.626 

m/m 

0.657 

1H/1H 

0.688 

m/m 

0.719 

m/m 

0.750 

N/1L 

0.621 

N/1H 

0.653 

1L/1H 

0.686 

N/1L 

0.716 

N/1L 

0.748 

N/2H 

0.620 

N/1L 

0.652 

N/1L 

0.684 

1L/1H 

0.714 

N/N 

0.744 

N/1H 

0.618 

N/2H 

0.650 

N/2H 

0.681 

N/2H 

0.712 

N/2H 

0.743 

Table  2.6  Sensitivity  to  U(x*,y°) 


for  losing  both  UAVs  associated  with  a  higher  probability  of  correct  identification 
and  neutralization  of  the  enemy. 


2-4-3  Sensitivity  to  7 x  and  7 y.  The  sensitivities  to  the  risk  parameters  for 
x  and  y  are  given  in  Tables  2.7  and  2.8,  respectively.  These  parameters  describe  the 
commander’s  risk  tolerance  in  the  single  attribute  space  for  x  and  y  given  complement 
attributes  at  the  maximum  (y*  and  x*).  Changes  in  these  parameters  change  the 
shape  of  the  utility  curve,  especially  at  the  margins. 

As  7X  changes  from  -0.2  to  0.1,  the  optimal  decision  changes  from  sending  only 
a  strike  UAV  at  high  altitude  to  a  strike  UAV  at  high  altitude  and  a  sensor  UAV  at 
low  altitude.  This  change  is  due  to  the  change  in  risk  attitude  from  risk  seeking  to 
risk  averse.  If  the  commander  is  risk  averse  in  the  enemy  destruction  attribute,  the 
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lx=- 

0.8 

lx=~ 

0.5 

lx=~ 

0.2 

7*=0-l 

7x=0.4 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

N/1H 

0.682 

N/1H 

0.685 

N/1H 

0.688 

1L/1H 

0.695 

1L/1H 

0.705 

m/m 

0.676 

1H/1H 

0.682 

1H/1H 

0.688 

m/m 

0.694 

m/m 

0.700 

N/1L 

0.673 

N/1L 

0.678 

1L/1H 

0.686 

N/1H 

0.691 

N/1L 

0.697 

N/2H 

0.670 

1L/1H 

0.678 

N/1L 

0.684 

N/1L 

0.690 

N/1H 

0.695 

1L/1H 

0.670 

N/2H 

0.675 

N/2H 

0.681 

N/2H 

0.687 

N/2H 

0.693 

Table  2.7  Sensitivity  to  7 x 


utility  of  ordering  a  strike  without  higher  sensor  accuracy  would  be  less  likely  due 
to  uncertainty  about  the  enemy  presence. 


7 y=- 

0.3 

7^=0.001 

7^=0. 3 

7y=0.6 

7^=0. 9 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

Se/St 

U 

N/1H 

0.648 

N/1H 

0.668 

N/1H 

0.688 

N/1H 

0.721 

N/N 

0.768 

N/2H 

0.636 

m/m 

0.657 

1H/1H 

0.688 

1H/1H 

0.720 

N/1H 

0.755 

1H/1H 

0.631 

N/2H 

0.656 

1L/1H 

0.686 

1L/1H 

0.718 

m/m 

0.750 

N/2L 

0.629 

1L/1H 

0.653 

N/1L 

0.684 

N/N 

0.716 

1L/1H 

0.748 

1L/1H 

0.620 

N/1L 

0.651 

N/2H 

0.681 

N/1L 

0.716 

N/1L 

0.745 

Table  2.8  Sensitivity  to  7 y 


As  7 y  changes  from  0.6  to  0.9  the  risk  of  losing  a  UAV  outweighs  the  benefit 
of  possible  enemy  destruction  according  to  the  commander’s  risk  attitude. 


2.5  Conclusions 

A  notional  decision  situation  consisting  of  UAV  assignment  involving  two  de¬ 
pendent  attributes  was  assessed.  A  procedure  for  developing  a  multiattribute  utility 
copula  [3]  was  implimented  and  modified  to  include  an  approximation  of  the  parame¬ 
ter  5,  measuring  the  dependence  of  the  attributes,  to  Spearman’s  rank  correlation,  p 
[15].  This  is  the  first  application  of  the  multiattribute  utility  copula  to  a  military  de¬ 
cision  problem. The  utility  dependence  between  attributes  required  a  multiattribute 
model  able  to  treat  such  dependence  issues.  The  sensitivity  analysis  presented  in  this 
chapter  shows  the  changes  in  the  optimal  alternatives  based  on  perturbations  of  the 
assessed  parameters.  This  is  an  important  step  in  the  decision  process  and  cannot 
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be  overlooked.  This  research  can  be  used  to  inform  decision  makers  of  the  impact 
of  their  risk  attitudes  and  value  tradeoffs  to  problems  involving  military  decision 
analysis,  and  alter  the  tactics,  techniques  and  procedures  (TTPs). 
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3.  An  Application  of  Copulas  to  UAV  Route  Planning 

3. 1  Introduction 

The  previous  chapter  examines  a  commander’s  decision  involving  a  single  target 
with  a  fixed  location.  This  allows  for  a  thorough  demonstration  of  the  effect  of  the 
copula  structure  to  the  decision  situation,  but  fails  to  account  for  the  effect  of  enemy 
movement  on  the  search  and  destroy  capabilities  of  UAVs.  This  chapter  explores  a 
notional  decision  where  a  commander  is  trying  to  plan  an  optimal  route  for  a  pair 
of  UAVs  searching  for  a  moving  target  through  various  types  of  terrain. 

The  search  for  a  moving  target  appears  in  many  sources  of  previous  litera¬ 
ture.  Stone  and  Kadane  [39]  classify  the  search  for  a  target  governed  by  Markovian 
movement  as  one  of  detection,  where  the  searcher  is  attempting  to  maximize  the 
probability  of  detection  by  time  t,  where-abouts,  where  the  goal  is  to  detect  the 
target  by  time  t  or  guess  the  whereabouts  at  time  t,  or  surveillance,  where  the  goal 
is  to  only  be  able  to  accurately  guess  the  location  of  the  target  at  time  t.  They 
argue  that  most  research  focuses  on  detection  or  whereabouts  searches  and  continue 
to  present  solution  techniques  to  the  whereabouts  search  type  problem. 

A  continuation  of  this  research  is  the  addition  of  multiple  searchers  by  Santos 
[34]  where  the  addition  of  multiple  searchers  for  a  single  target  exhibiting  stochastic 
movement  is  shown  to  be  computationally  difficult  and  require  the  use  of  advanced 
heuristics,  of  which  Genetic  Algorithms  (GAs)  perform  very  well.  Dell  et  al.  [11] 
further  examine  the  use  of  many  heuristic  decision  types  on  a  randomly  moving 
enemy  using  multiple  searchers  along  a  constrained  path.  In  their  research,  GA 
performs  very  well  among  the  several  heuristics  examined.  They  also  introduce 
the  concept  of  Bayesian  updates  for  the  location  of  the  enemy  given  a  probability 
of  non-detection  at  a  discrete  time  step.  Hollinger  et  al.  [18]  study  the  concept 
of  coordinated  versus  non-coordinated  search  patterns  through  Bayesian  updated 
target  search  for  a  non- adversarial  enemy  utilizing  a  partially  observable  markov 
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decision  process  to  define  the  movement.  Their  research  centers  around  a  group  of 
semi-autonomous  robots  searching  an  indoor  space. 

Optimal  policies  of  active  binary  classification  using  Bayesian  updates  to  label 
all  data  on  a  graph  are  studied  extensively  by  Garnett  et  al.  [14].  This  research 
focuses  on  a  one  or  two-step  look  ahead  policy  for  determining  the  next  location  to 
investigate  on  the  graph.  In  this  approach,  the  utility  is  equal  to  the  probability 
of  detection,  fitting  with  the  detection  search  type  presented  by  Stone  and  Kadane 

[39]. 

The  search  problem  presents  a  similarity  with  the  Vehicle  Routing  Problem 
(VRP),  which  has  a  large  body  of  work  focusing  on  the  use  of  GAs  to  solve.  Ombuki 
[29]  and  Nagata  [27]  present  improvements  on  the  GA  for  VRPs  with  time  windows 
for  delivery  by  modifying  the  crossover  and  mutation  steps.  A  problem  with  the 
current  GA  types  is  the  introduction  (and  subsequent  deletion)  of  infeasible  children 
for  a  VRP  formulation,  rectified  through  the  use  of  a  Hybrid  GA  presented  by  Prins 
[31]  that  does  not  create  any  infeasible  children  in  the  crossover  or  mutation  steps. 
Yang  and  Yuan  [46]  present  an  improvement  of  the  GA  where  mutations  involve 
random  swaps  of  genes  and  inversion  of  genes  in  specific  chromosomes.  This  is 
largely  the  form  used  in  the  GA  applied  in  this  research. 

Weinstein  [45]  and  Schumacher  [35]  present  the  UAV  cooperative  search  prob¬ 
lem  as  a  VRP  with  time  windows  formulation  through  the  use  of  a  Mixed  Integer 
Linear  Program.  Each  uses  a  set  of  tasks  including  location,  classification,  attack 
and  battle  damage  assessment  to  be  performed  by  multiple  UAVs.  Weinstein  [45] 
allows  the  UAVs  to  avoid  early  detection  by  coordinating  movement  into  specific 
areas  of  the  map. 

Research  into  UAV  routing  is  also  not  lacking.  Russel  and  Larnont  [33]  present 
an  improved  GA  specific  to  UAV  mission  planning  that  performs  better  on  standard 
test  problems  than  previous  GA.  Bertuccclli  et  al.  [9]  examines  a  UAV  task  assign¬ 
ment  problem  with  uncertainty,  where  there  is  Bayesian  updating  of  target  locations 
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and  tasks  include  location  classification  and  strikes.  Shima  et  al.  [37]  introduce  a 
matrix  representation  of  chromosomes  to  reduce  computing  time  in  a  multi-UAV 
task  assignment  GA.  This  problem  is  viewed  as  a  combinatorial  optimization  prob¬ 
lem  that  is  NP-Hard.  Jin  et  al.  [20]  incorporates  a  target  response  to  the  cooperative 
search  and  task  assignment  problem. 

Pohl  and  Lamont  [30]  examine  the  routing  of  UAV  swarms,  or  large  groups  of 
semi-autonomous  UAVs.  Employing  GA,  they  force  several  UAVs  to  track  several 
targets  while  staying  within  distance  constraints. 

Terrain  covering  is  another  search  type  presented  by  Ablavsky  [5]  where  dis¬ 
tribution  of  effort  between  multiple  UAVs  provides  for  a  complete  covering  of  a  2D 
space  using  geometric  movement  capabilities  of  the  aircraft.  In  this  form,  multiple 
coverage  on  the  same  set  of  UAVs  may  be  performed,  but  penalized  through  the 
objective  function.  Another  example  of  the  coverage  of  a  2D  map  is  presented  by 
Wang  [44];  this  dissertation  includes  Bayesian  updates  and  DTMC  defined  target 
movement. 

Abdclhafiz  et  al.  [6]  present  several  instances  of  the  multi-objective  UAV  mis¬ 
sion  planning  problem  where  the  introduction  of  risk  to  UAVs  as  a  deterrent  to 
scheduling  actions  creates  a  more  complex  solution  space.  This  tradeoff  between 
goals  of  coordinated  search  and  destruction  and  risk  mitigation  provides  the  back¬ 
bone  of  the  use  of  decision  analysis  in  this  research. 

The  rest  of  this  chapter  is  organized  into  sections.  Section  3.2  discusses  the 
methodological  approach  for  the  heuristic  to  solve  this  problem.  Section  3.3  examines 
a  scenario  where  an  enemy  is  attempting  to  evade  detection.  Section  3.4  examines 
an  enemy  trying  to  travel  from  one  area  of  the  map  to  another.  Section  3.5  discusses 
relevant  conclusions  to  be  drawn  from  this  analysis. 
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3.2  Methodology 

In  this  model,  enemy  movement  is  assumed  to  be  Markovian.  The  distribution 
of  enemy  location,  E(t),  is  entirely  characterized  through  the  use  of  an  initial  belief  in 
location  and  a  transition  matrix  P.  A  discrete  time  markov  chain  serves  to  model  the 
enemy  movement  at  each  time  period.  As  the  model  progresses  through  time  periods, 
Bayesian  updates  define  a  new  vector  of  probabilities  for  location  of  the  enemy  based 
on  input  from  previous  time  epochs  as  described  in  Section  3.2.1  and  demonstrated 
in  Section  3.3.4.  The  probability  of  enemy  location  is  considered  a  state,  s,  and  the 
decision  maker  can  send  UAVs  to  a  location,  changing  the  subsequent  utility  and 
probability  distributions  of  enemy  locations.  This  is  a  Markov  Decision  Process  with 
a  utility  function  to  characterize  the  outcomes. 

The  construction  of  the  utility  copula  used  to  measure  a  commander’s  risk 
preferences  among  alternatives  uses  the  same  procedures  as  outlined  in  Chapter  2. 
This  section  introduces  the  notion  of  time  into  the  decision  situation  through  the 
use  of  Bayesian  updating  and  provides  an  outline  of  the  GA  used  to  search  the 
alternatives  for  the  solution  with  the  highest  expected  utility.  The  utility  function 
used  for  this  research  is  Equation  2.4. 

In  this  analysis  a  maximum  of  five  time  periods  is  examined  and  it  is  assumed 
that  a  UAV  has  a  standard  operating  window  of  sufficient  length  to  allow  uncon¬ 
strained  flight  through  the  area  of  operations  without  a  distance  consideration.  A 
specific  area  on  the  map  may  be  searched  multiple  times,  providing  that  there  is  an 
increased  utility  associated  with  doing  so.  If  the  probability  of  enemy  detection  is  suf¬ 
ficiently  low,  based  on  Bayesian  updates,  this  is  a  way  to  avoid  the  enemy  entirely, 
avoiding  UAV  damage  and  reducing  the  probability  of  firing  at  a  low-probability 
target.  In  this  manner,  the  model  has  many  similarities  to  the  standard  VRP,  but 
without  the  normal  constraints  on  distance  or  number  of  visits. 
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3.2. 1  Bayesian  Updates.  The  process  of  Bayesian  updating  is  seen  through¬ 
out  the  literature  as  a  way  to  update  belief  in  enemy  locations  based  on  known  in¬ 
formation  after  each  time  epoch.  In  this  manner,  the  future  assignment  of  UAVs 
to  search  areas  on  a  map  are  influenced  by  searches  in  past  time  epochs.  This  is 
done  using  the  conditional  prior  belief  in  enemy  location  and  the  observations  for 
the  locations  using  equation  3.1 


P(E\V) 


P(V\E) 

E„  P(E\Vn)P(Vn) 


P(V). 


(3.1) 


This  calculates  the  posterior  belief  in  enemy  presence  at  that  node.  Subsequent  nodes 
are  calculated,  and  updates  occur  on  the  nodes  that  were  not  observed  based  on 
these  calculations.  UAVs  seeing  the  enemy  or  being  destroyed  each  have  a  different 
effect  in  updating  the  probability  of  enemy  presence  in  an  area  on  the  map,  and 
subsequent  decision  tree  nodes  reflect  this  probabilistic  belief.  Sample  calculations 
of  the  Bayesian  updates  are  shown  in  Section  3.3.4. 


3.2.2  Genetic  Algorithm.  GA  is  a  heuristic  approach  to  searching  the  al¬ 
ternatives  for  the  best  solution  to  a  fitness  function.  This  is  done  through  the  use 
of  a  chromosome  representing  each  alternative.  The  locations  of  the  nodes  visited 
at  a  certain  time  are  genes  along  this  chromosome.  Mutations  are  random  replace¬ 
ment  of  these  genes  (nodes)  with  another  feasible  gene.  Crossovers  are  meant  to 
mimic  natural  reproduction  and  provide  a  splicing  of  the  genes  associated  with  two 
chromosomes.  The  population  size  of  a  genetic  algorithm  ensures  that  only  the  best 
solutions  from  each  iteration  are  kept  and  allowed  to  mutate  or  crossover. 

To  effectively  search  for  the  alternatives  with  the  highest  expected  utility,  a 
GA  is  used.  Use  of  this  heuristic  approach  is  shown  in  the  literature  to  be  one 
of  the  most  efficient  approaches  to  solving  the  VRP  [27,  29,  33,  34,  37].  For  this 
specific  application  of  GA  to  the  VRP,  mutation  procedures  as  outlined  by  Yang 
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and  Yuan  [46]  is  used,  in  conjunction  with  standard  crossover  procedures.  A  matrix 
representation  provides  computational  simplicity  [37]. 

The  expected  utility  is  used  as  the  fitness  function  for  the  GA.  Chromosomes 
are  sorted  by  the  expected  utility  of  their  paths  and  only  the  best  chromosomes  are 
kept  for  the  next  iteration  of  the  GA. 

Since  there  are  no  constraints  on  the  number  of  times  that  a  node  may  be 
visited,  the  crossover  operation  is  simplified  to  a  switching  of  genes  along  the  chro¬ 
mosomal  representation  for  each  path.  Figures  3.1  and  3.2  show  an  example  of  a 
crossover  operation  performed  on  two  chromosomes. 
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Figure  3.1 

Chromosomes  A  and  B  prior  to  crossover 
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Figure  3.2  Chromosomes  A  and  B  after  crossover 


Two  types  of  mutations  are  used.  The  inversion  operation  selects  a  section  of 
chromosome  and  reverses  the  order  of  the  genes  along  that  section.  Figure  3.3  shows 
a  chromosome  before  and  after  inversion. 

The  second  type  of  mutation  is  the  random  replacement.  In  this  mutation  a 
random  number  (between  1  and  3)  of  genes  are  replaced  with  randomly  chosen  areas 
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Figure  3.3  Chromosome  which  has  undergone  inversion  muta¬ 
tion 

of  the  map  to  test  for  better  solutions.  Figure  3.4  shows  a  random  replacement  on 
3  genes  in  a  chromosome. 
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Figure  3.4  Random  replacement  on  3  genes 


Due  to  the  lack  of  constraints  and  the  subsequent  lack  of  infeasible  children,  a 
GA  performs  well  in  searching  the  decision  space  for  this  model.  As  the  complexity 
of  the  model  increases,  through  scope  or  constraints,  other  heuristics  should  be 
considered. 

3.3  Notional  Scenario  1 

3.3.1  Decision  Context.  A  notional  2D  map  space  is  comprised  of  three 
different  terrain  types.  Each  terrain  type  provides  bonuses  or  penalties  in  the  form 
of  modified  probabilities  for  the  enemy  forces  and  UAV  searchers  capabilities  (e.g. 
survivability,  accuracy).  Table  3.1  and  Table  3.2  represent  the  terrain  types  and 
adjacency  list  for  the  fully  connected  hexagonal  map  space.  Figure  3.5  shows  a 
graphical  representation  of  the  area  map.  Node  numbers  are  the  areas  of  the  graph, 
and  the  letters  represent  the  terrain  type  associated  with  that  area. 
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Node 

Terrain 

Node 

Terrain 

Node 

Terrain 

Node 

Terrain 

1 

Hill 

10 

Mountain 

19 

Plains 

28 

Plains 

2 

Mountain 

11 

Mountain 

20 

Hill 

29 

Hill 

3 

Mountain 

12 

Mountain 

21 

Hill 

30 

Hill 

4 

Mountain 

13 

Hill 

22 

Hill 

31 

Plains 

5 

Mountain 

14 

Hill 

23 

Hill 

32 

Plains 

6 

Mountain 

15 

Mountain 

24 

Hill 

33 

Plains 

7 

Hill 

16 

Mountain 

25 

Plains 

34 

Plains 

8 

Hill 

17 

Mountain 

26 

Plains 

35 

Plains 

9 

Mountain 

18 

Mountain 

27 

Plains 

36 

Plains 

Table  3.1  Terrain  type  by  node 


Node 

Adjacency 

Node 

Adjacency 

1 

1, 

2,  7 

19 

13, 

14, 

19, 

20, 

25 

2 

1, 

2,  3,  7,  8 

20 

14, 

15, 

19, 

20, 

21, 

25, 

26 

3 

2, 

3,  4,  8,  9 

21 

15, 

16, 

20, 

21, 

22, 

26, 

27 

4 

3, 

4,  5,  9,  10 

22 

16, 

17, 

21, 

22, 

23, 

27, 

28 

5 

4, 

5,  6,  10,  11 

23 

17, 

18, 

22, 

23, 

24, 

28, 

29 

6 

5, 

6,  11,  12 

24 

18, 

23, 

24, 

29, 

30 

7 

1, 

2,  7,  8,  13 

25 

19, 

20, 

25, 

26, 

31 

8 

2, 

3,  7,  8,  9,  13,  14 

26 

20, 

21, 

25, 

26, 

27, 

31, 

32 

9 

3, 

4,  8,  9,  10,  14,  15 

27 

21, 

22, 

26, 

27, 

28, 

32, 

33 

10 

4, 

5,  9,  10,  11,  15,  16 

28 

22, 

23, 

27, 

28, 

29, 

33, 

34 

11 

5, 

6,  10,  11,  12,  16,  17 

29 

23, 

24, 

28, 

29, 

30, 

34, 

35 

12 

6, 

11,  12,  17,  18 

30 

24, 

29, 

30, 

35, 

36 

13 

7, 

8,  13,  14,  19 

31 

25, 

26, 

31, 

32 

14 

8, 

9,  13,  14,  15,  19,  20 

32 

26, 

27, 

31, 

32, 

33 

15 

9, 

10,  14,  15,  16,  20,  21 

33 

27, 

28, 

32, 

33, 

34 

16 

10 

',  11,  15,  16,  17,  21,  22 

34 

28, 

29, 

33, 

34, 

35 

17 

11 

,  12,  16,  17,  18,  22,  23 

35 

29, 

30, 

34, 

35, 

36 

18 

12 

:,  17,  18,  23,  24 

36 

30, 

35, 

36 

Table  3.2  Adjacency  listing  for  scenario  1 


The  commander  must  plan  the  UAV  flight  path  for  two  UAVs  through  four 
time  periods.  Each  aircraft  has  a  list  of  four  nodes  to  visit  and  the  subsequent  utility 
is  calculated  using  the  probabilities  and  utility  as  described  in  the  later  sections. 

3.3.2  Information.  In  this  notional  scenario,  assume  that  the  enemy  begins 
at  node  21  and  has  movement  described  by  a  P  matrix  created  with  the  following 
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Figure  3.5  Graphical  representation  of  Area  Map 


arbitrary  mechanism  to  define  belief  in  enemy  movement;  each  adjacent  node  is  coded 
as  a  1  if  plains,  2  if  hills,  and  3  if  mountains.  This  serves  as  a  mechanism  for  differing 
probabilities  of  enemy  movement,  UAV  survivability,  and  UAV  accuracy.  The  sum 
of  all  adjacencies  is  added  in  this  manner  and  the  probability  of  traveling  to  each 
adjacent  node  is  the  number  coded  divided  by  the  sum  of  all  adjacent  codes.  In  this 
manner,  the  calculation  of  the  transition  vector  associated  with  node  21,  which  is 
adjacent  to  15,  16,  20,  21,  25,  and  26,  is  calculated  as  follows;  each  node  is  assigned 
a  number  associated  with  its  terrain  type,  which  sums  to  14.  The  probability  of 
the  enemy  staying  at  node  21  would  be  2/14.  This  method  of  coding  transition 
probabilities  is  notional. 

Probabilities  for  the  UAV  performance  are  dependent  on  location  and  presence 
of  enemy.  A  list  of  these  probabilities  can  be  found  in  Table  3.3.  If  there  is  an  enemy, 
a  strike  results  in  enemy  neutralization  with  a  probability  of  0.8.  If  there  is  no  enemy, 
a  strike  results  in  collateral  damage  with  the  same  probability. 
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Distinction 

Terrain 

Probability 

Accuracy 

Mountains 

0.85 

Accuracy 

Hills 

0.9 

Accuracy 

Plains 

0.95 

Survivability  Given  Enemy 

Mountains 

0.7 

Survivability  Given  Enemy 

Hills 

0.8 

Survivability  Given  Enemy 

Plains 

0.9 

Survivability  Given  No  Enemy 

Mountains 

0.9 

Survivability  Given  No  Enemy 

Hills 

0.95 

Survivability  Given  No  Enemy 

Plains 

0.98 

Table  3.3  Probability  adjustments  for  terrain  type 


3.3.3  Alternatives  and  Preferences.  The  commander  must  plan  the  flight 
paths  for  both  UAVs  and  the  alternatives  are  all  possible  flight  plans.  An  assigned 
restriction  is  that  UAVs  cannot  operate  in  the  same  map  location  in  the  same  time 
period.  The  attributes  X  and  Y  are  similar  to  those  described  in  Chapter  2,  however, 
the  scaling  on  the  attributes  is  slightly  different.  The  X  attribute  is  defined  as  enemy 
neutralization  and  is  defined  on  x  G  {0,  0.5, 1}.  A  0  represents  a  friendly  fire  incident, 
a  0.5  represents  an  unsuccessful  strike  or  no  strike,  and  a  1  represents  complete  enemy 
neutralization.  The  Y  attribute  represents  the  number  of  UAVs  returning  after  the 
mission  and  is  defined  on  y  e  {0,0.5, 1}.  A  0  represents  no  UAVs  returning,  a  0.5 
represents  1  UAV  returning,  and  a  1  represents  2  UAVs  returning  after  the  mission 
is  complete. 

The  notional  assessed  parameters  for  the  commander  in  this  decision  situation 
are  as  follows.  The  value  of  no  strike  but  all  UAVs  returning  is  U(x°,y*)  =  0.15. 
The  value  of  a  succsessful  strike  but  both  UAVs  lost  is  U (x*,  y°)  =  0.4.  The  notional 
assessed  ranked  correlation  coefficient  is  p  =  0.5.  The  respective  risk  aversion  coef¬ 
ficients  are  7 x  =  —1.2  and  =  1.3.  The  risk  aversion  coefficients  reflect  a  decision 
maker  who  is  risk  seeking  in  gains  and  risk  averse  in  losses  as  prescribed  by  Tversky 
and  Kahneman  [40]. 
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After  assessing  these  parameters  for  this  decision  situation,  the  multiattribute 
utility  copula  parameters  5,  lx,  ly,  a,  and  b  are  determined  by  using  Equations  2.5, 
2.6,  2.7,  2.8,  and  2.9.  The  following  values  are  obtained: 

6  =  3.2646,  a  =  1.6363,  b  =  -0.6363,  lx  =  0.4805,  ly  =  0.6333. 


The  multiattribute  utility  copula  C(x,y )  is  found  by  substituting  into  Equation 
2.4.  Figure  3.6  shows  the  multiattribute  utility  function  and  Figure  3.7  shows  the 
isopreference  curves. 


Isopreference  Cuives 


Figure  3.6  Multiattribute  utility 

surface 


Figure  3.7  Multiattribute 
isopreference  curves 


3.3.4  Sample  Calculations.  The  calculations  for  utility  and  Bayesian  up¬ 
dates  for  a  specific  path  are  demonstrated  in  this  section.  Using  a  sample  alternative 
from  the  notional  scenario,  the  calculations  are  performed  for  the  first  time  period 
for  the  path  where  UAV1  visits  nodes  27,  21,  20,  and  15  and  UAV2  visits  nodes  22, 
20,  26,  and  10  at  each  time  period  respectively.  The  posterior  probabilities  for  the 
first  time  period  are  calculated  using  the  following  equations: 
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For  UAV  1: 


P(S(  1))  =  P(E)P(S\E)  +  P(E)P(S\E)  = 
0.071  *  0.9  +  0.929  *  0.98  =  0.9743, 
P{V{1)\S)  =  P(E)P(S\E)P(V\E)/P(S)  +  P(E)P(S\E)P(V\E)/P(S)  = 
0.071  *  0.95  *  0.9/0.9743  +  0.929  *  0.98  *  0.05/0.9743  =  0.1094, 
P(E(1)\VS)  =  P(E)P(S\E)P(V\E)/(P(S)P(V\S)) 
=  0.071  *  0.9  *  0.95/(0.9743  *  0.1094)  =  0.5703. 


For  UAV  2: 


P(S{ 2))  =  0.143  *  0.8  +  0.857  *  0.95  =  0.9286, 
P(V(2)\S)  =  0.143  *  0.8  *  0.9/0.9286  +  0.857  *  0.95  *  0.1/0.9286  =  0.1986, 
P(E(2)\VS)  =  0.143  *  0.8  *  0.9/(0.9286  *  0.1986)  =  0.5581. 


The  utility  of  the  first  time  period  is  calculated  for  each  outcome.  For  example, 
the  utility  in  the  case  where  both  survive  and  see  the  enemy  would  be  given  by: 


P(5(1))P(5(2))(P(V(1)|5)P(V(2)|5)* 
max(P(E(l)\VS)(P(Hit\E)UM (3,  3)  +  P(NoHit\E)UM(2,3))  + 
P(E(l)\VS)(P(Hit\E)UM(l,3)  +  P(NoHit\E)UM(2,3)), 
P(E(2)\VS)(P(Hit\E)UM(3,3)  +  P(NoHit\E)UM(2,3))  + 
P(E(2)\VS)(P(Hit\E)UM(l,3)  +  P(NoHit\E)UM(2,3))). 
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The  other  outcomes  are  calculated  similarly  and  the  resulting  cumulative  ex¬ 
pected  utility  for  time  period  1  is  found.  After  this  calculation,  Bayesian  updates 
are  calculated  for  all  outcomes  where  the  enemy  was  not  found  (either  by  UAV  de¬ 
struction  or  lack  of  sensor  accuracy).  The  outcome  where  both  aircraft  survived  but 
did  not  see  the  enemy  would  be  calculated  in  the  following  manner.  Node  27  would 
be  updated  to 

p(Fm)  =  P(E)P(S\E)P(V\E) _ = 

1  1  P(E)P{S\E)P{V\E)  +  P(E)P(S\E)P{V\E) 

0.071  *  .9  *  .05 

- =  0.0037. 

0.071  *  .9  *  .05  +  0.929  *  0.98  *  .95 

The  resulting  new  enemy  probability  vector  is  given  by: 

PE= [00000000000000  0.2675  0.2675  0  0 

0  0.1784  0.1784  0.0154  000  0.0892  0.0037  000000000], 

This  vector  is  then  multiplied  by  the  transition  matrix  P  to  give  the  enemy  locations 
for  the  next  time  period  for  the  subsequent  outcomes.  This  process  is  repeated  for 
all  possible  outcomes  at  each  time  period. 

3.3.5  Results.  The  alternative  with  the  highest  expected  utility  is  to  send 
UAV  1  to  area  16,  21,  15,  and  15  and  send  UAV  2  to  22,  20,  21,  and  21.  Table  3.4 
shows  the  resulting  expected  utility  of  the  five  best  alternatives  found  by  the  GA. 


Rank 

UAV  1 

UAV  2 

Utility 

1 

16,  21,  15,  15 

22,  20,  21,  21 

0.425 

2 

16,  21,  15,  16 

22,  20,  21,  15 

0.424 

3 

16,  21,  15,  21 

22,  20,  21,  15 

0.423 

4 

16,  21,  15,  21 

22,  20,  21,  20 

0.422 

5 

16,  21,  15,  21 

22,  20,  21,  19 

0.420 

Table  3.4  Best  alternatives  for  Notional  Scenario  1 
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The  best  result  is  shown  in  Figure  3.8.  In  this  figure  the  red  nodes  represent 
those  visited  by  UAV  1  and  the  blue  nodes  represent  those  visited  by  UAV  2. 


(a)  Time  Epoch  1 


(b)  Time  Epoch  2 


(c)  Time  Epoch  3  (d)  Time  Epoch  4 

Figure  3.8  Nodes  visited  by  each  UAV  at  each  time  period  in 
scenario  1 

There  are  multiple  visits  to  several  nodes  on  this  list.  As  time  advances,  the 
enemy  location  probability  in  any  area  of  the  map  decreases.  Thus,  in  the  later 
time  epochs,  the  UAVs  may  search  an  area  multiple  times  to  avoid  searching  an  area 
with  a  high  chance  of  target  misidentification  as  a  result  of  a  low  prior  probability 
of  enemy  presence. 

3.3.6  Sensitivity  Analysis.  The  following  section  details  the  sensitivity 
analysis  to  the  assessed  parameters  for  constructing  the  multiattribute  utility  copula 
as  well  as  the  probabilities  of  survival  and  accuracy  for  the  UAV  sensors.  The 
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sensitivites  to  p,  U(x*,y°),  U(x°,y *),  7X,  ^y,  P(S\E),  P(V\E),  and  the  number  of 
time  steps  evaluated  by  the  utility  model  are  examined. 

3.3.6. 1  Sensitivity  to  p.  The  paramater  p  is  translated  into  5  to  de¬ 
scribe  the  corresponding  tradeoffs  curve  between  x  and  y.  Changes  in  the  parameter 
act  in  the  same  way  on  the  utility  function  as  in  Chapter  2. 

The  model  is  not  very  sensitive  to  changes  in  the  correlation  between  the 
utilities  of  X  and  Y.  The  areas  searched  by  each  UAV  does  not  change  for  p  e 
[—0.4, 1.0].  At  p  =  —0.41  the  paths  change  to  send  UAV  1  to  16,  21,  21,  and  15 
and  to  send  UAV  2  to  22,  20,  20,  and  10.  This  change  is  due  to  the  utility  of  the 
intermediate  outcomes  (those  not  at  the  extreme  margins)  changing.  The  parameter 
p  affects  the  areas  in  the  center  of  the  utility  copula,  thus  the  outcome  of  x  —  0.5, 
and  y  =  0.5  will  be  changed.  As  this  change  propegates  through  the  decision  tree, 
this  will  change  the  overall  utility  of  the  alternatives,  and  can  change  the  answer.  If 
these  outcomes  are  unlikely,  no  changes  in  the  optimal  alternative  will  appear. 

3. 3. 6. 2  Sensitivity  to  U(x* ,y°)  andU(x°,y*).  The  parameters  U(x*,  y°) 
and  U ( x° ,  y*)  define  the  corner  points  of  the  maximum  marginal  utility  functions  and 
are  directly  assessed  from  the  decision  maker.  U(x*,y°)  defines  the  utility  of  finding 
and  destroying  the  enemy  target  without  the  destruction  of  a  UAV.  The  optimal 
alternative  does  not  change  for  small  perturbations  of  this  value;  it  remains  constant 
for  U (x*,y°)  G  [0,  0.5].  For  U (x*,  y°)  >  0.5,  changes  in  the  areas  of  the  map  searched 
by  the  UAVs  change  slightly.  UAV  1  still  visit  16,  21,  15,  and  15.  However,  UAV  2 
now  visits  22,  20,  20,  and  16.  This  change  is  due  to  rather  large  changes  in  the  shape 
of  the  utility  function  at  the  maximum  margin  for  U(y).  This  changes  the  utilities 
associated  with  y  —  0  and  y  =  0.5. 

The  parameter  U(x°,y*)  is  the  utility  of  eradicating  a  friendly  force,  but  re¬ 
taining  both  UAVs.  This  paramater  causes  more  changes  in  the  optimal  alternative. 
The  results  of  changes  are  shown  in  Table  3.5. 
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Parameter 

UAV  1 

UAV  2 

U(x° 

,y*] 

=0.05 

16, 

21, 

15, 

15 

22, 

20,  20, 

16 

U(x° 

,y*) 

=0.15 

16, 

21, 

15, 

15 

22, 

20,  21, 

21 

U(x° 

,y*] 

=0.25 

16, 

21, 

21, 

15 

22, 

20,  20, 

10 

U(x° 

,y*] 

=0.35 

16, 

21, 

21, 

15 

22, 

20,  20, 

21 

U(x° 

,y*) 

=0.45 

16, 

21, 

21, 

21 

22, 

20,  20, 

20 

Table  3.5  Sensitivity  to  U(x°,y*)  for  scenario  1 

Changing  U(x°,y*)  alters  the  assignment  of  the  later  searched  nodes,  where 
the  probability  of  UAV  loss  is  greater  relative  to  the  probability  of  finding  the  enemy. 
UAV  survival  is  dependent  upon  natural  flight  problems  and  enemy  presence. 

3. 3. 6. 3  Sensitivity  to  and  Changes  to  7 x  and  change  the 

shape  of  the  utility  curve  at  the  maximum  margins,  since  this  is  a  Class  1  utility 
copula.  The  result  is  to  change  the  entire  shape  of  the  utility  function  and  can 
change  interior  as  well  as  marginal  utility  values.  The  corner  points  remain  fixed,  as 
those  are  only  changed  through  the  parameters  U(x*,y°)  and  U(x°,y*). 

The  parameter  7^  is  stable,  with  no  changes  occurring  in  the  range  of  7^  G 
[—0.2, —2.2],  At  7a,  =  0.8,  as  the  decision  maker  changes  to  being  risk  prone  in 
the  attribute  x,  the  search  path  for  UAV  2  changes  to  22,  20,  21,  and  10,  a  very 
slight  change.  At  7 x  =  —3.2,  the  search  patterns  for  both  UAVs  change.  UAV  1 
now  searches  16,  21,  21,  and  15  and  UAV  2  searches  22,  20,  20,  and  10.  This  is 
because  the  utility  of  the  intermediate  values  for  attribute  x  are  higher,  resulting  in 
less  penalty  for  missing  the  enemy  with  a  strike. 

The  parameter  is  also  stable,  with  only  changes  to  the  last  nodes  searched 
being  made.  The  optimal  alternative  for  =  0.3  is  UAV  2  searching  22,  20,  20, 
and  16,  with  no  change  to  UAV  1,  and  the  optimal  alternative  for  =  3.3  is  UAV 
2  searching  22,  15,  21,  20,  again  with  no  changes  for  UAV  1.  This  reflects  the  result 
that  the  priority  nodes  do  not  change  based  on  changes  to  the  parameter  7^  and 
that  the  model  is  not  very  sensitive  to  changes  in  this  parameter. 
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3. 3. 6. 4  Sensitivity  to  Assessed  Probabilities.  The  assessed  probabil¬ 
ities  for  survival  and  accuracy  affect  the  likelihood  of  outcomes  from  the  decision 
tree  for  each  alternative,  and  thus,  can  have  large  effects  on  the  expected  utility.  As 
survivability  decreases,  UAVa  tend  towards  avoiding  enemies  with  larger  penalties 
(like  mountainous  terrain).  As  accuracy  decreases,  the  posterior  probability  of  en¬ 
emy  location,  and  the  accuracy  of  the  Bayesian  updates  for  subsequent  time  epochs 
is  diminished. 

The  probability  of  survival  is  subject  only  to  small  changes  at  a  lower  projected 
value.  Changing  the  probability  of  survival  given  enemy  and  mountains  to  0.6,  0.7 
for  hills  and  0.8  in  plains  resulted  in  a  UAV  2  path  of  22,  20,  20,  and  16,  with  no 
change  in  the  path  to  UAV  1,  indicating  that  only  the  last  time  epoch  was  changed. 
However,  upon  increasing  the  survival  to  0.8  for  mountains,  0.9  for  hills,  and  0.99 
for  plains,  the  path  with  the  highest  expected  utility  for  UAV  2  was  22,  15,  21  and 
20,  with  no  change  to  UAV  1.  This  shows  that  increasing  the  probability  of  survival 
allowed  for  a  more  thorough  search  of  mountain  terrain  (area  15)  and  hills  (area  20). 

The  accuracy  of  the  sensors  for  UAV  is  stable  as  it  is  decreasing,  but  results 
in  a  small  change  as  it  is  increasing.  At  an  accuracy  of  0.9  for  mountains,  0.95  for 
hills  and  0.99  for  plains,  the  path  for  UAV  2  shifted  to  node  16  for  the  fourth  time 
epoch,  ensuring  a  more  thorough  search  of  mountainous  terrain. 

3. 3. 6. 5  Sensitivity  to  the  Planning  Horizon.  This  section  describes 
the  sensitivity  of  the  model  to  the  number  of  time  steps  that  the  commander  has 
to  plan  for  UAV  movement.  In  longer  planning  horizons,  it  is  possible  for  the  UAV 
to  use  Bayesian  updating  and  the  properties  of  Markovian  movement  to  ensure  that 
mountainous  areas  of  the  map  do  not  have  to  be  searched  with  high  probability  of 
enemy  activity  (where  survival  probabilities  are  lower),  whereas  in  shorter  planning 
horizons,  it  may  be  more  beneficial  to  go  to  the  locations  with  the  highest  probability 
of  enemy  activity. 
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Shortening  the  planning  horizon  to  2  time  steps  results  in  the  alternative  with 
the  highest  expected  utility  being  sending  UAV  1  to  16  and  21  and  UAV  2  to  15 
and  20.  This  is  the  first  sensitivity  analyzed  in  this  paper  to  effect  a  change  in  the 
locations  searched  in  the  first  time  epoch.  The  UAVs  are  sent,  in  this  instance,  to 
the  locations  with  the  highest  probability  of  enemy  activity  in  each  time  epoch. 

At  a  planning  horizon  of  3  time  steps,  the  path  of  UAV  1  changed  to  20,  21, 
and  15  and  UAV  2  became  16,  16,  21.  Again,  drastic  changes  from  the  2  and  4  time 
step  planning  horizons  are  present,  and  some  avoidance  (area  15)  of  enemy  is  taking 
place  in  earlier  time  steps  to  ensure  a  higher  probability  of  UAV  survival. 

At  a  planning  horizon  of  5  time  steps,  the  optimal  path  of  UAV  1  is  16,  21, 
21,  15  and  15  and  UAV  2  is  22,  20,  20,  10,  and  21.  This  shows  some  change  from 
the  model  at  4  time  steps,  but  the  changes  occur  in  later  time  epochs.  It  is  clear 
that  as  the  planning  horizon  is  made  longer  it  becomes  advantageous  in  utility  to 
not  directly  search  areas  with  the  highest  enemy  probability  as  soon  as  possible  in 
every  case. 

3.4  Notional  Scenario  2 

3-4- 1  Decision  Context.  In  the  second  notional  scenario  considered  by  this 
analysis,  the  enemy  is  considered  to  have  a  goal.  After  starting  at  area  31,  two  time 
steps  have  passed  and  it  is  assumed  that  the  enemy  will  attempt  to  reach  a  safe  zone 
at  node  6.  Table  3.6  represents  adjacency  list  for  the  directed  hexagonal  map  space. 

The  adjacency  list  for  this  space  is  greatly  reduced,  as  the  enemy  will  only 
move  towards  their  goal,  and  no  longer  backtrack  or  stay  in  an  area  for  more  than 
one  time  unit.  The  probability  of  enemy  movement  into  any  area  is  unknown,  so  it 
is  assumed  that  they  will  move  into  any  adjacent  node  (according  to  the  adjacency 
list)  with  equal  probability  to  all  other  adjacent  nodes.  The  information  that  was 
received  on  the  enemy  was  delayed,  so  two  time  steps  have  taken  place  before  the 


3-18 


Node 

Adjacency 

Node 

Adjacency 

1 

2 

19 

13,  14,  20 

2 

3 

20 

14,  15,  21 

3 

4 

21 

15,  16,  22 

4 

5 

22 

16,  17,  23 

5 

6 

23 

17,  18,  24 

6 

- 

24 

18 

7 

1,2,8 

25 

19,  20,26 

8 

2,  3,  9 

26 

20,  21,  27 

9 

3,  4,  10 

27 

21,  22,  28 

10 

4,  5,  11 

28 

22,  23,  29 

11 

5,  6,  12 

29 

23,  24,  30 

12 

6 

30 

24 

13 

7,  8,  14 

31 

25,  26,  32 

14 

8,  9,  15 

32 

26,  27,  33 

15 

9,  10,  16 

33 

27,  28,  34 

16 

10,  11,  17 

34 

28,  29,  35 

17 

11,  12,  18 

35 

29,  30,  36 

18 

12 

36 

30 

Table  3.6  Adjacency  listing  for  scenario  2 


search  can  begin.  The  search  will  be  conducted  for  three  time  steps,  with  sensitivities 
performed  on  increasing  or  decreasing  that  planning  horizon. 

3-4-2  Results.  The  alternative  with  the  highest  expected  utility  is  to  send 
UAV  1  to  area  27,  14,  and  16  and  send  UAV  2  to  26,  15,  and  22.  Table  3.7  shows 
the  resulting  expected  utility  of  3  of  the  best  alternatives  found  by  the  GA. 


Rank 

UAV  1 

UAV  2 

Utility 

1 

27,  14,  16 

26,  15,  22 

0.530 

2 

27,  15,  16 

26,  14,  22 

0.529 

3 

27,  15,  15 

26,  14,  22 

0.528 

Table  3.7  Best  alternatives  for  Notional  Scenario  2 

The  best  result  is  shown  in  Figure  3.9.  In  this  figure  the  red  nodes  represent 
those  visited  by  UAV  1  and  the  blue  nodes  represent  those  visited  by  UAV  2. 
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(a)  Time  Epoch  1 


(b)  Time  Epoch  2 


Figure  3.9  Nodes  visited  by  each  UAV  at  each  time  period  in 
scenario  2 

It  should  be  noted  that  there  are  multiple  ways  to  get  to  certain  areas  of  the 
map  while  taking  different  amounts  of  time.  For  example,  area  26  can  be  reached  by 
moving  directly  from  31  or  moving  from  31  to  25  to  26.  Area  26  is  visited  at  time 
step  2,  while  there  is  still  a  probability  of  hireling  the  enemy  there. 

3-4-3  Sensitivity  Analysis.  The  following  section  details  the  sensitivity 
analysis  to  the  assessed  parameters  for  constructing  the  multiattribute  utility  copula 
as  well  as  the  probabilities  of  survival  and  accuracy  for  the  UAV  sensors.  The 
sensitivites  to  p,  U(x*,y°),  U(x°,y*),  ”/x,  7y,  P(S\E),  P(V\E),  and  the  number  of 
time  steps  evaluated  by  the  utility  model  are  examined. 
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3-4-3. 1  Sensitivity  to  p.  The  parameter  p  is  used  to  calculate  the 
parameter  5  describing  the  corresponding  tradeoffs  curve  between  x  and  y.  This 
measure  defines  the  shape  of  the  utility  surface,  and  as  seen  in  Chapter  2,  can 
greatly  impact  a  decision. 

In  this  scenario,  there  are  no  changes  to  the  decision  made  based  on  an  increase 
to  the  parameter  p\  however,  the  optimal  paths  change  if  p  is  decreased  even  slightly. 
At  p  =  0.4,  the  optimal  paths  become  27,  21,  and  16  for  UAV  1  and  26,  14,  and  10 
for  UAV  2.  This  change  is  due  to  changes  in  the  intermediate  utility  values  rippling 
through  the  probability  calculations  in  the  decision  tree. 

3. 4- 3. 2  Sensitivity  to  U(x*,y°)  and,U(x°,y*).  The  parameters  U(x*,  y°) 
and  U (a;0,  y*)  are  the  utility  corner  points  for  the  maximum/minimum  value  of  x  and 
minimum/maximum  value  of  y.  Changes  to  these  parameters  can  change  the  entire 
shape  of  the  utility  curve. 

In  this  scenario,  the  optimal  decision  does  not  change  with  changes  to  U(x*,y°), 
even  extreme  changes.  As  U(x°,y*)  is  increased,  to  0.35,  the  optimal  paths  change 
to  27,  21,  and  16  for  UAV  1  and  26,  14,  28  for  UAV  2.  This  change  reflects  a  greater 
importance  on  UAVs  returning,  and  thus,  an  larger  avoidance  of  mountainous  areas 
of  the  map  than  the  general  result. 

3. 4- 3. 3  Sensitivity  to  7 x  and  7y.  The  parameters  7 x  and  7^  define 
the  shape  of  the  utility  surface  at  the  maximum  margins  for  Class  1  utility  copulas. 
This  shape  change  affects  the  rest  of  the  utility  surface  shape  as  well,  but  is  more 
pronounced  at  the  maximal  marginal  utility. 

This  analysis  showed  no  changes  to  even  extreme  values  in  7^.  The  optimal 
path  remained  fixed  for  all  7 x  G  [—2.8, 4.2],  The  parameter  7^  is  stable  in  increases, 
and  the  optimal  decision  only  change  for  extreme  decreases.  At  7^  =  —2.7  the  paths 
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chang  to  UAV  1  to  26,  21,  and  16  and  UAV  2  to  27,  14,  and  10.  This  reflects  a  more 
cavalier  attitude  by  the  commander  in  UAV  loss. 

3. 4 -3. 4  Sensitivity  to  Assessed  Probabilities.  As  in  notional  scenario 
1,  the  assessed  probabilities  define  the  likelihood  of  the  resulting  outcomes  for  each 
alternative.  Changes  in  survivability  and  accuracy  are  examined  in  this  analysis. 

Survivability  affects  the  likelihood  of  UAV  to  be  shot  down  by  enemy  forces 
based  on  the  probability  of  enemy  activity  and  the  terrain  type  that  they  are  flying 
in.  At  a  survivability  of  0.6  for  mountains,  0.7  for  hills,  and  0.8  for  plains,  the 
optimal  alternative  changes  to  sending  UAV  1  to  26,  21  and  16  and  UAV  2  to  27,  14, 
and  10.  The  optimal  alternative  does  not  change  with  increases  to  the  survivability 
of  the  drones. 

3. 4 .3. 5  Sensitivity  to  the  Planning  Horizon.  As  seen  in  scenario  1, 
the  planning  horizon  can  effect  how  the  UAVs  search  the  area,  effectively  ensuring 
that  probabilities  of  enemy  presence  in  later  time  epochs  are  higher  at  specific  areas, 
or  sacrificing  this  to  find  the  enemy  more  quickly.  This  scenario  is  no  different. 

At  a  2-time  step  planning  horizon,  the  optimal  alternative  become  sending 
UAV  1  to  26  then  21  and  sending  UAV  2  to  27  then  14.  This  represents  the  areas 
with  the  highest  probability  of  enemy  presence  and  lowest  probability  of  UAV  loss 
(no  mountainous  terrain  is  searched).  This  changes  for  the  4-time  step  planning 
horizon,  where  UAV  1  goes  to  26,  14,  22,  10  and  UAV  2  goes  to  27,  15,  28,  11.  In 
this  scenario,  there  is  a  distinct  ’Tunneling”  of  enemy  activity  into  nodes  10  and  11, 
to  increase  the  chance  of  finding  the  enemy  at  later  time  epochs. 

3.5  Conclusions 

The  research  presented  in  this  chapter  is  a  natural  MAUT  extenstion  to  the 
classic  search  and  destroy  theoretical  problem  addressed  in  Section  4.1.  This  research 
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applies  current  trends  in  Decision  Analysis  to  address  a  commander’s  risk  preferences 
associated  with  the  problem. 

In  this  chapter,  two  notional  decision  situations  were  evaluated  consisting  of  a 
time  dependent  UAV  search  for  a  markovian  moving  target.  The  attributes  of  enemy 
neutralization  and  UAV  survival  were  used  to  create  a  multiattribute  utility  copula, 
and  a  GA  was  used  to  search  the  alternatives  for  the  alternative  with  the  highest 
expected  utility. 

The  concept  of  time  as  it  applies  to  a  search  using  multiattribute  utility  is 
an  interesting  prospect.  This  provided  an  ability  for  the  UAV  to  route  the  enemy 
into  areas  where  the  search  would  be  either  more  profitable  or  less  dangerous.  This 
aspect  of  model  shows  where  the  utility  copula  formulation  allows  for  tradeoffs  to 
dictate  the  paths  the  UAVs  take  in  such  a  way  to  increase  the  overall  expected  utility 
of  the  search. 
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4-  Conclusion 

This  research  provides  an  original  approach  to  the  UAV  search  and  destroy  problem 
by  employing  multiattribute  utility  theory.  Multiattribute  utility  theory  is  useful  in 
any  situation  where  a  decision  involves  competing  objectives.  In  the  case  where  these 
these  objectives  are  utility  dependent  on  each  other,  multiattribute  utility  copulas 
provide  a  mechanism  to  model  the  dependence. 

This  thesis  has  demonstrated  a  new  class  of  multiattribute  utility  functions 
with  a  Class  1  Archimedian  copula  structure.  Two  scenarios  provided  applications 
of  the  decision  model  and  various  operational  results  are  derived.  The  work  of  this 
thesis  is  the  initial  step  in  examining  the  decision  process  facing  commanders  in  a 
UAV  search  and  destroy  situation. 

Several  examples  of  future  research  in  the  application  of  multiattribute  utility 
copulas  to  UAV  search  and  destroy  problems  exist.  As  the  number  of  UAVs  at  the 
commanders  disposal  increase,  so  does  the  size  and  complexity  of  the  alternatives 
and  outcomes.  This  research  limited  the  number  of  UAVs  to  2;  however,  there  are 
often  more  UAVs  available,  it  would  there  fore  be  worthwhile  to  examine  the  use 
of  more  than  two  UAVs.  Another  possible  extension  is  to  add  anti-air  defenses, 
enemy  safe  zones,  or  structures  to  the  area  map  in  scenario  2.  These  adjustments 
add  to  the  complexity  of  the  problem,  but  allow  for  a  more  realistic  interpretation 
of  the  decision  situation.  These  variables  would  require  careful  analysis  as  to  the 
benefit  and  cost  to  target  destruction  or  UAV  survival  before  implementation.  Other 
extensions  to  consider  are  the  introduction  of  more  terrain  features,  such  as  roads 
or  towns  and  the  separation  of  collateral  damage  into  a  separate  attribute. 

This  research  is  useful  in  situations  where  there  is  a  long  term  planning  ob¬ 
jective  that  requires  careful  consideration  of  risk  attitdues.  The  use  of  this  research 
on  many  common  decisions  could  serve  to  update  the  TTPs  associated  with  those 
problems.  Other  applications  that  may  benefit  from  the  use  of  multiattribute  utility 
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copulas  include  the  search  for  a  downed  pilot,  where  resources  and  success  serve  as 
attributes,  or  the  area  of  cyber  warfare,  where  risk  of  detection  is  competing  against 
infiltration  of  an  enemy  network. 
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Appendix  A.  Quad  Chart 


Using  Multiattribute  Utility  Copulas  in  Support 
of  UAV  Search  and  Destroy  Operations 


Research  Goal 

The  goal  of  this  research  is  to  apply  an  emerging 
form  of  utility  function,  the  multiattribute  utility 
copula,  to  UAV  search  and  destroy  mission 
planning  operations.  This  research  informs 
decision  makers  and  analysts  with  respect  to 
tactics,  techniques,  and  procedures  employed  in 
such  missions.  Increasing  UAV  operations  tempo 
suggests  such  research  becoming  increasingly 
relevant  to  the  warfighter. 


Stationary  Enemy 

Utility  copula  is  used  to  help  decision  makers  determine  the  type,  quantity,  and  altitude  of  UAVs  to  send  to 
strike  a  stationary  target.  A  commanders  risk  attitude  helps  to  determine  the  tradeoff  between  UAV 
exposure  to  enemy  and  the  ability  to  strike  a  target. 

Moving  Enemy 

Utility  copula  examines  the  impact  of  attribute  dependeence  on  the  optimal  routing  of  UAVs  in  a  contested 
environment  when  performing  a  search  and  destroy  mission  against  a  Markovian  target  with  Bayesian 
updates. 


Capt  Beau  Nunnally  AFIT/ENS 
Committee 


Moving  Enemy  Model  Formulation 


p  -  Ranked  correlation  coefficient,  used  to  solve  for  6 


Scenario  1 

This  scenario  involved  the  search  for  an  enemy  with 
random  movement,  starting  in  node  21. 

Movement  was  defined  using  ratios  with  movement 
to  mountains  being  most  likely  followed  by  hills  and 


Scenario  2 

This  scenario  involved  the  search  for  an  enemy  with 
goal  oriented  movement,  starting  in  node  31,  the 
enemy  is  attempting  to  reach  node  6. 

Movement  is  equally  likely  to  any  of  the  nodes  above 
the  current  location. 


Stationary  Enemy  Model  Formulation 


Decision  Diagram 

Commander  must  decide  whether  to  send  2, 1  or  0 
UAVs  at  a  target.  The  UAVs  may  be  loaded  with  a 
sensor  package  or  a  strike  package,  and  act  as 
independent  sensors.  Upon  reaching  the  enemy  the 
commander  must  make  a  decision  on  whether  to  fire 
or  not.  The  outcomes  of  these  decisions  each  have  a 
utility  as  prescribed  by  a  multiattribute  utility  copula. 


Multiattribute  Utility  Copula 


Assumptions  of  utility  independence  (as  is  typically  done)  does  not  provide  the  correct  answer.  Since  the 
multilinear  utility  requires  the  assumption  of  conditional  independence  among  attributes,  the  resulting 
utility  on  one  attribute  would  not  update  the  utility  of  another.  The  copula  form  allows  for  a  mutual  utility 
dependence  structure,  which  more  accurately  reflects  the  decision  maker's  preferences. 

Sensitivity  Analysis 

A  sensitivity  analysis  is  performed  on  all  assessed  parameters  showing  a  robust  decision  model . 
Conclusions 

This  research  can  be  used  to  inform  decision  makers  of  the  impact  of  their  risk  attitudes  and  value  tradeoffs 
to  problems  involving  military  decision  analysis,  and  alter  the  tactics,  techniques,  and  procedures  associated 
with  these  decisions. 


Multiattribute  Utility  Function 


Conclusions 

This  research  provides  an  original  approach  to 
the  UAV  search  and  destroy  problem  by 
employing  multattribute  utility  theory. 

Future  Work 

Increase  the  number  of  UAVs  actively  searching 
the  decision  space 

Add  anti-air  defense,  enemy  safe  zones, 
structures,  or  other  terrain  features  to  the  area 
map. 

Applications 

This  research  can  be  applied  to  several 
problems  including  the  search  for  a  downed 
pilot,  long  term  attack  scenarios,  and  cyber 
warfare. 
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